E3: Emotions, Engagement and Educational Games

This study is an investigation of ways to collect student engagement and gameplay data from a digital educational game called Quantum Spectre in order to understand student engagement in such digital environments, and the effect of certain affective states on student gameplay behavior. Proposed study participants are elementary school students, 5 graders, who will play the game over the course of multiple class sessions. Previous research findings suggest that there is an interesting inter-relation between frustration and confusion that requires more attention; the indices of frustration and confusion are influenced by the amount of external support provided. Based on these initial findings, the proposed dissertation experiment will concentrate on significant patterns of frustration and confusion along with their effect on student gameplay and further engagement with the environment. STATEMENT OF THE PROBLEM AND STUDY SIGNIFICANCE Digital educational games have become a popular means of instruction in recent years (Mayo, 2009; O’Neil, Wainess, & Baker, 2005; Rodrigo et al., 2008). Many educational concepts (e.g., science, technology, engineering, and mathematics education concepts) are taught and practiced through digital educational game environments. This instructional approach is mainly justified with the realization that games can naturally motivate students to engage with the environments and learn (Prensky, 2001; Kapp, 2012). From the researcher point of view, games provide students with a safe space for failure and confidence to persist. From the perspective of educational establishments, online games offer a unique advantage of simultaneous accessibility for thousands of children along with a customizable learning pace and ability to follow students’ learning trajectories. Overall, scientists consider games as a potentially powerful tool for learning (FAS, 2006). Lack of student engagement can be a threat to the learning environment; disengaged students may not take full advantage of the learning opportunities offered through these settings. Academic affect is one factor that can either benefit or undermine students’ engagement and learning. Previous research shows there is a complex interaction between affect and learning (Baker, D'Mello, Rodrigo, & Graesser, 2010). Moreover, affective states trigger different results in different human-computer interaction environments (Rodrigo & Baker, 2011) and depending on the order of affective states, the impact may be negative or positive. Many researchers acknowledge the importance of understanding students’ affective responses to success and failure. Two relevant areas of research on this topic are Angela Duckworth’s work on grit (2007) and Carol Dweck’s work on selfefficacy (1985; 1991). According to Dweck (2002), a learner’s goal orientation (i.e., beliefs about one’s abilities and the effectiveness of their effort) may influence their affective response to the success and failure they experience within an environment. As Duckworth (2007) identifies it, grit or persistence is about “sticking with things over the very long term until you master them,” which includes overcoming negative experiences of frustration, confusion, and failure. Persistence is currently being researched as it is considered to be a key factor in college completion or completion of similar academic long-term goals (Duckworth, 2007; Duckworth & Seligman, 2005). Well-designed digital educational game environments should be able to provide support for high levels of frustration that could be detrimental for student engagement while developing persistence as students meet new challenges within the game. There have been many studies concerned with student affective states and their impact on student engagement and motivation in intelligent tutoring systems (ITS) (Rodrigo et al., 2012). However, this promising work has not yet been fully extended to digital educational game environments (O’Rourke, Haimovitz, Ballwebber, Dweck & Popovic, 2014). Therefore, in the current study I address this gap by looking into student behavior, affect and the effect of emotions on student learning within digital educational game environments. The use of digital educational games is becoming widespread, however, its technological design is not on the same level with intelligent tutoring systems where the environment promotes learning through adaptive guidance. My research is most relevant to the areas of research in affective computing, learning through digital games for learning, and game development. This will inspire game developers to design games that will be more responsive to negative displays of affect to keep students engaged in their environments. In fact, if we are able to detect negative manifestations of certain affective states, game developers will be able to incorporate this detection feature into future digital education game designs and incorporate recommender systems into educational game environments. Hence, it is worthwhile to continue research that informs game design to include sensorless detection (i.e., not based on data collected from external sensor devices or other extremely obtrusive methods of data collection such as heart rate monitors, eye trackers or skin conductance) of affect, which will provide students with only necessary hints to persist and will not interrupt their beneficial exploration stage. This affect detection system might ameliorate students’ negative perceptions of their own abilities in the fields of science and mathematics by guiding them through their confusions and frustrations associated with the learning environment. However, in order to support learning and increase academic goal orientation while students are engaged in digital game environments for learning, we need to understand student motivation and the emotions that affect them. Students’ affective states play a critical role in their performance. Potentially negative emotional states such as confusion and frustration are more crucial to investigate, since emotional variability can be one of the moderating factors of success and failure in the struggle to overcome barriers in goal attainment. Moreover, frustration and confusion are two affective states that are suggested to lead to boredom state. As literature suggest (Baker, D’Mello, Rodrigo, & Graesser, 2010), it is better to be frustrated than bored since boredom leads to disengagement and makes it much harder to bring students back to engagement and concentration from boredom emotional state. Therefore, I believe that if developed well, digital educational games with adaptive support systems may become one of the key ways to assisting students to push past confusion and frustration and develop persistence regardless of their goal orientation. The better and more precise our research findings, the more sophisticated and helpful our educational games will become. While there have been studies looking into the effects of frustration or/and confusion on student learning and possibility of reducing frustration (Baker et al., 2010; Hone, 2006; Klein, Moon, & Picard, 2002; McQuiggan, Lee, & Lester, 2007), almost none of these studies have looked into whether there is an interaction and order to the pattern of frustration and confusion along with their influence on student engagement with the environment (e.g., concentration or boredom patterns). Kort, Reilly, and Picard (2001) attempted a model of confusion to frustration transitions but their empirical evidence did not support their hypothesized model. In addition, Perkins and Hill (1985) have hypothesized that frustration leads to boredom but their analysis did not allow for such a conclusion since they illustrated association instead of temporal or sequential connection. Yet another study investigated the decay rate of certain cognitiveaffective states, however, it did not concentrate on patterns of occurrence but rather on the temporal and tripartite classification of affect (D’Mello & Graesser, 2011). This study by D’Mello and Graesser (2011) has informed the design of this proposed study. Thus, while there have been many attempts at investigating frustration and confusion sequential patterns, there seems not to be empirical evidence on this subject either due to inappropriate analysis method or inconclusive results. Thus, my work will contribute to current research on frustration and confusion by using a sequential pattern mining algorithm on categorical affect sequences in order to identifying sequential patterns and possible interdependency that need to be avoided in order to keep students engaged in digital game environments for learning and make sure students have uninterrupted opportunity for learning. RESEARCH QUESTIONS, METHODOLOGY AND SOLUTIONS I believe that to help students learn better and be invested in their own education, we need to understand the motivation and the emotions that affect them while going through learning processes. My dissertation will be focusing on certain emotions, frustration and confusion, manifested while playing an educational game (science learning game). In addition to investigating both frustration and confusion in EG environments, I will be evaluating the relationship between frustration and confusion and the combination of these two affective states that has destructive effect on learning or engagement. Some of the research questions that this study will be investigating are as follows: • What is the relationship between frustration and confusion and when is each beneficial or negative? • Is there a sequential pattern in the occurrence of frustration and confusion or is there no temporal pattern • What is the relationship between frustration/confusion and student success? • Is there a significant difference on student engagement when employing self-report method of affect data collection vs. unobtrusive field observation method? • Does the ratio of frustration to confusion states significantly change in relation to the amount of support available or is there no correlation between the amount of support provided and students’ frustration and confusion? For example, are low-risk environments (e.g., adult/peer assistance available) related to lower amounts of confusion and no to low amount of frustration? • Do students resume educational gameplay after being in a frustrated emotional state? If so, when and under what conditions do they resume their gameplay?